Skip to content

fix: Layer Deployment Permission Issue#2356

Merged
mndeveci merged 14 commits intoaws:developfrom
CoshUS:issue/go-zip-permission
Nov 7, 2020
Merged

fix: Layer Deployment Permission Issue#2356
mndeveci merged 14 commits intoaws:developfrom
CoshUS:issue/go-zip-permission

Conversation

@CoshUS
Copy link
Contributor

@CoshUS CoshUS commented Nov 6, 2020

Issue #, if available:
#2344
Why is this change necessary?
Lambda needs read permission as owner on artifacts.

How does it address the issue?
Changed Windows mocked permission to 755, mimicking Unix system.
Previously, it was changed from default 644 to 005 to give execute permission for go artifacts. Owner and group owner permission bits were removed assuming Lambda itself (not the user executing the code) do not need permission on the artifacts. However, as regression issue #2344 stated, it does require read permission on the Layer artifacts.

There should be no security concern as owner and group owner are set and used by Lambda.

What side effects does this change have?
Owner and group owner permission bits will be set to 7 and 5 respectively.

Did you change a dependency in requirements/base.txt?
If so, did you run make update-reproducible-reqs

Checklist:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@CoshUS CoshUS changed the title Issue/go zip permission Layer Deployment Permission Issue Nov 6, 2020
Copy link
Contributor

@mndeveci mndeveci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, and I tested it on Windows and confirmed it is working.

@awsjeffg
Copy link

awsjeffg commented Nov 7, 2020

@CoshUS THANK YOU!

@mndeveci mndeveci merged commit 71be4f8 into aws:develop Nov 7, 2020
@OK-Apps
Copy link

OK-Apps commented Nov 12, 2020

still getting same problem even with version 1.9.0 :

$ sam --version
SAM CLI, version 1.9.0


UPDATE_FAILED            AWS::Lambda::Function    StagingKinesisSenderEx   Layer conversion
                                                  tended                   failed: Some files do
                                                                           not have read
                                                                           permissions; (Service:
                                                                           AWSLambdaInternal;
                                                                           Status Code: 400;
                                                                           Error Code: InvalidPar
                                                                           ameterValueException;
                                                                           Request ID: 4db54727-3
                                                                           8f7-41c9-a648-655361a2
                                                                           2882; Proxy: null)
UPDATE_FAILED            AWS::Lambda::Function    StagingKinesisPutRecor   Layer conversion
                                                  d                        failed: Some files do
                                                                           not have read
                                                                           permissions; (Service:
                                                                           AWSLambdaInternal;
                                                                           Status Code: 400;
                                                                           Error Code: InvalidPar
                                                                           ameterValueException;
                                                                           Request ID: a05a655b-0
                                                                           cfb-4f57-9025-2ab308b2
                                                                           068d; Proxy: null)
UPDATE_ROLLBACK_IN_PRO   AWS::CloudFormation::S   Staging-StagingKinesis   The following
GRESS                    tack                                              resource(s) failed to
                                                                           update: [StagingKinesi
                                                                           sSenderExtended, Stagi
                                                                           ngKinesisPutRecor


@CoshUS
Copy link
Contributor Author

CoshUS commented Nov 12, 2020

@OK-Apps It seems this could be caused by the wrong permission set in the existing resources using the previous version of SAM CLI. Can you try manually deleting those two resources and re-deploy again?

@CoshUS CoshUS changed the title Layer Deployment Permission Issue fix: Layer Deployment Permission Issue Nov 12, 2020
@OK-Apps
Copy link

OK-Apps commented Nov 12, 2020

@OK-Apps It seems this could be caused by the wrong permission set in the existing resources using the previous version of SAM CLI. Can you try manually deleting those two resources and re-deploy again?

@CoshUS by "resource" do you mean the lambda functions or the lambda layers ? (or both?)
and,
in case if it is only about lambda functions (and not layers) -
if I was able to successfully do a 'sam deploy; from a non-Windows machine (mac) - so does it count as same effect
as "manually delete"? - so would we expect if we now try again from a windows machine with 1.9.0

thank you!

@CoshUS
Copy link
Contributor Author

CoshUS commented Nov 13, 2020

@OK-Apps

do you mean the lambda functions or the lambda layers ? (or both?)

My hunch would be only layers, but try deleting or overriding both.

if I was able to successfully do a 'sam deploy; from a non-Windows machine (mac) - so does it count as same effect
as "manually delete"? - so would we expect if we now try again from a windows machine with 1.9.0

If your deployment on Mac overrides the existing layer artifact with correct permission bits, then it should be the same as deleting and creating a new one.

Thanks!

@OK-Apps
Copy link

OK-Apps commented Nov 18, 2020

@CoshUS

an update of current state of affairs - everyone in a team has SAM CLI 1.9.0
when doing 'sam build' & 'sam deploy' from Mac - all works ok
when doing same from various Windows 10 machines - getting the error:

Layer conversion failed: Some files do not have read permissions;
(Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException;
Request ID: 4df2bf50-f584-4e85-9538-5173260b072e; Proxy: null)

@OK-Apps
Copy link

OK-Apps commented Nov 18, 2020

@CoshUS so puzzled how is it even possible - if things take place at server-side- at AWS,
why it behaves differently when triggering it from Mac VS Windows ?

@OK-Apps
Copy link

OK-Apps commented Nov 18, 2020

@CoshUS
regarding layers themselves - some layers we did not even use sam to create them - they been created/uploaded "manually" into AWS console

so what we need to do with those layers? (if we need to do anything with layers to fix this)

@CoshUS
Copy link
Contributor Author

CoshUS commented Nov 18, 2020

A few clarifying questions:

regarding layers themselves - some layers we did not even use sam to create them - they been created/uploaded "manually" into AWS console

Were the layers created with sam deployed using SAM CLI 1.7.0 or 1.8.0 on Windows?

an update of current state of affairs - everyone in a team has SAM CLI 1.9.0
when doing 'sam build' & 'sam deploy' from Mac - all works ok
when doing same from various Windows 10 machines - getting the error:

Are these deployments using the same layers and the same Lambdas?


how is it even possible - if things take place at server-side- at AWS

Lambda is based on Linux which uses unix permission bits. When packaging a layer artifact, Windows does not set permission bits in the zip file which can cause missing permission errors when uploaded and unpacked into Lambda. We changed this behavior in 1.7.0 and forced all file permissions to be 005 when packaged in Windows. This had unintended consequences in deploying layers that can result in Layer conversion failed: Some files do not have read permissions; We then changed the permission to 755 to be more permissible in SAM CLI 1.9.0.

If your layers are created manually in Windows, it should have the default 644 permission and not cause read permission issue.

@kurax
Copy link

kurax commented Nov 18, 2020

I'm using the lastest release of SAM CLI as 1.10.0 and this issue still remains.
I've deleted all my deployed functions by comment them out in template.yml, then uncomment try to recreate and got this:

Layer conversion failed: Some files do not have read permissions; (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: 194fed35-a3b8-4ed5-8b17-d93823171bbc; Proxy: null)

Any ideas?

UPDATE: The LayerVersion creation was successful which would fail on 1.8.0, I just can't attach it to functions with Layers configuration, although I'm able attach it on console page, but if you did that, this function later won't able to update with SAM by the same exception.

@OK-Apps
Copy link

OK-Apps commented Nov 19, 2020

HI @CoshUS
thank you for update - and yes it is our understadning s well ... we have observed the issue with Windows permisions in GoLang lambda
and seen them fixed in 1.7 ... but then started seeing this "layers issue" with Python lambdas
and now even with 1.9 - still experiencing it (on Windows machines !!)
despite the fact that
"layers are created manually in Windows, it should have the default 644 permission and not cause read permission issue."

some of our Python layers been manually created on Linux because it had to be done that way - i.e. we cannot just "re-create under WIndows"

So what is the way forward from this situation now?

Is any version of SAM CLI expected to fix/solve this? - I mean to be able to continue to use SAM CLI from this state of affairs - when got layers that been created manually under Linux because they have to be done under Linux

@CoshUS
Copy link
Contributor Author

CoshUS commented Nov 19, 2020

@OK-Apps @kurax
Thanks for the information. Created issue #2392 to track this problem.

Is any version of SAM CLI expected to fix/solve this?

SAM CLI 1.6.2 does not have the Go permission fix. Can you give that a try?

@CoshUS
Copy link
Contributor Author

CoshUS commented Nov 19, 2020

@OK-Apps
Are you able to provide a sample template that reproduces the issue?

@OK-Apps
Copy link

OK-Apps commented Nov 28, 2020

@CoshUS

@OK-Apps
Are you able to provide a sample template that reproduces the issue?

pls see attached template.zip

also,
yes it worked ok under SAM CLI 1.5/1.6

thank you!
template.yml.zip

@esscott2112
Copy link

esscott2112 commented Jan 21, 2021

I am having the exact same issue with Window machine. SAM deployment works with Linux but not Windows for all the reasons listed. When will this be fixed in the SAM so my development team can SAM deploy from windows machine?
I am having the issue with Python Layers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

Comments